Test Design Patterns for Embedded Systems

Test suites for embedded systems are typically created from scratch using different, often inadequate methods. In consequence, industry branches dealing with software-intensive embedded systems have to cope with quality problems, even though test processes are particularly time-consuming and costly. Based on an evolving model-based testing methodology we introduce test design patterns for simplifying and accelerating the test specification process. Making use of patterns, complex test objectives are covered with lower effort and test cases contain fewer design errors. 1 Test Specification for Embedded Systems A reasonable and potentially succesful test specification method for embedded systems implies understanding of the system under test (SUT) and the test engineering areas. For that purpose we firstly characterise the domain of embedded systems according to our knowledge and then we introduce the methods and test patterns for testing them. Our goal is to support the dynamic black-box software testing. 1.1 Testing within the Embedded Systems Domain Software in embedded systems is gradually becoming of even higher importance than hardware [SZ06] due to the creation of new innovative functions based on software and due to the implementation of formerly mechanically or electronically integrated functions in embedded software. The definition of embedded systems has often been a source of misunderstanding among experts. Relating to such systems we mean mainly embedded software of a certain functional complexity, presenting hybrid behaviour (due to continuous and discrete signals) and constrained by temporal conditions. Addi1 This work is partially supported by the ITEA2 within the project: Deployment of Model-based Technologies to Industrial Testing, http://www.d-mint.org/. 2 J. Zander-Nowicka, A. Marrero Pérez, I. Schieferdecker, Z. R. Dai tionally, in this paper we focus on systems within the automotive domain, where several software variations are used to address the same functionality in different models. Thus, the configuration (i.e. parameterization) of the test system is a complex task. As a result, the test method for embedded systems has to cope with all those obstacles to provide the efficiency and effectiveness. An automated, but systematic and consistent test specification method is needed. The test execution should proceed automatically. The proposed technique must handle discrete and continuous signals, as well as timing constraints. We assume that the requirements are available in forms of a textual specification and an executable model of the SUT exists. Moreover, we support testing on the model level, when neither code, nor hardware exists yet. By that we start testing already in the design phase. This allows for resource and cost reduction during the entire software development process. Finally, the test reusability is an important quality issue, especially if a number of similar SUT variants have to be tested over and over again. By supporting reusability, a concept of patterns emerges. A pattern is a general solution to a specific recurring design problem. It explains the insight and good practices that have evolved to solve the problem. It provides a concise definition of common elements, context, and essential requirements for a solution as discussed in [Ale79] [VS04] [Neu04] [PTD05]. Patterns were first used to describe constellations of structural elements in buildings and towns [Ale79]. Moreover, patterns contribute to such quality characteristics like maintainability and efficiency due to a common and understandable vocabulary for problem solutions that they provide [Neu04]. 1.2 Introduction to Test Patterns Test patterns [VS04] represent a form of reuse in the test development in which the essences of solutions and experiences gathered in testing are extracted and documented so as to enable their application in similar contexts that might arise in the future. Test patterns aim at capturing test design knowledge from past projects in a canonical form, so that future projects would benefit from it. The author of [Bert07] considers the efficacy-maximized test engineering and arguments that any good testing practices need to be collected by a systematic effort so as to organize recurring effective solutions into a catalogue of test patterns, similarly to what is now a well established scheme for design approaches. Such a test pattern extraction [VS04] is the process of abstracting from existing problem-solution-benefit triples in the test developing process, to obtain patterns suitable for reuse in future contexts. Although the process of going through existing test artefacts and trying to identify patterns for later reuse might appear costly and unrewarding at the first sight, it pays off in a long term [VS04]. This argumentation especially applies to testing of embedded software, where reusability of some common test procedures is appreciated. The test paterrns provide means for test system developers to focus more on what to test and less on the notation itself. With this practice, they simplify the test development process, increase the level of automation and facilitate the understandability of the test. Test Design Patterns for Embedded Systems 3 The novelty of this paper reflects both: a new method for the test specification of embedded systems as well as a related concept supporting the test patterns extraction and application.